Congestion Control via Online Sampling

We consider the congestion-control problem in a communication network with multiple traffic sources, each modeled as a fullycontrollable stream of fluid traffic. The controlled traffic shares a common bottleneck node with high-priority cross traffic described by a Markovmodulated fluid (MMF). Each controlled source is assumed to have a unique round-trip delay. We wish to maximize a linear combination of the throughput, delay, traffic loss rate, and a fairness metric at the bottleneck node. We introduce an online sampling-based burst-level congestioncontrol scheme capable of performing effectively under rapidly-varying cross traffic by making explicit use of the provided MMF model of that variation. The control problem is posed as a finite-horizon Markov decision process and is solved heuristically using a technique called Hindsight Optimization. We provide a detailed derivation of our congestion-control algorithm based on this technique. The distinguishing feature of our scheme relative to conventional congestion-control schemes is that we exploit a stochastic model of the cross traffic. Our empirical study shows that our control scheme significantly outperforms the conventional proportionalderivative (PD) controller, achieving higher utlization, lower delay, and lower loss under reasonable fairness. The performance advantage of our scheme over the PD scheme grows as the rate variance of cross traffic increases, underscoring the effectiveness of our control scheme under variable cross traffic. Keywords—Communication networks, congestion control, traffic models, Markov-modulated fluid, Markov decision processes, online sampling.